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AMENDMENTS TO THE CLAIMS 

Prior to the present communication, claims 10-11, 15, 17-18, 20-22, 24-28, and 
30-31 were pending in the subject application. All of claims 10-11, 15, 17-18, 20-22, 24-28, and 
30-31 have been amended and claims 32-36 have been added herein. Thus, claims 10-11, 15, 
17-18, 20-22, 24-28, and 30-36 remain pending. This Listing of Claims will replace all prior 
versions, and listings, of claims in the application. 

Listing of Claims 

1-9. (Canceled) 

10. (Currently Amended) A computer-implemented method of capturing and 
treating content using a computer system having a processor, memory, and data storage 
subsystems, the computer-implemented method comprising the steps of : 

a) setting a mode of operation to a content capture mode for interpreting 
user stylus input for the purpose of selecting an on-screen region of a display, and 
receiving a path drawn by a user via the stylus as input , the path defining 
boundaries of the selected on-screen region of the [[a]] display, wherein pixels 
comprising one or more graphical elements are displayed in the selected on- 
screen region; 

b) capturing the pixels displayed within the selected on-screen region, and 
storing the captured pixels in an image file such that the image file is 
representative of only those pixels of the display within the selected on screen 



3788428vl 



Page 5 of 23 



Application No. 10/766,319 306171.01/MFCP.153380 

Response Filed 03/03/2010 

Reply to Office Action of: 10/05/2009 

c) receiving a user command to set the mode of operation to an annotation 
mode for interpreting the user stylus input for the purpose of annotating the one or 
more graphical elements and, in response to the user command, switching the 
mode of operation to the annotation mode and receiving an annotation drawn by 
the user on the display via the stylus , wherein the received annotation is 
implemented using a plurality of tools via a toolbar, the toolbar appearing after 
the selecting an on-screen region ; and 

d) obtaining context information for the one or more graphical elements 
by automatically applying text recognition to the annotation drawn by the user on 
the display via the stylus , and storing the results of the text recognition as context 
information via the computer system , 

wherein the context information is automatically stored in association with 
the image file. 

1 1 . (Currently Amended) The computer-implemented method according to 
claim 10, further comprising: 

determining a window associated with the selected on-screen region; 

retrieving an application interface having a Uniform Resource Identifier 
(URI) property from the determined window or a parent window of the 
determined window; and 

obtaining the URI property as additional context information, the 
additional context information being automatically stored in association with the 
image file. 
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12-14. (Canceled) 

15. (Currently Amended) The computer-implemented method of claim 10, 
further comprising: 

creating and storing a linking structure as the association between the 
image file and the context information. 

16. (Canceled) 

17. (Currently Amended) The computer-implemented method of claim 15, 
wherein the linking structure is incorporated in a file separate from the stored image file and the 
stored context information. 

18. (Currently Amended) The computer-implemented method of claim 15, 
wherein the linking structure includes at least one pointer pointing to the stored image file or the 
stored context information. 

19. (Canceled) 

20. (Currently Amended) The computer-implemented method of claim 10, 
wherein the context information is stored in such a manner as to be accessible to the [[a]] user for 
performing at least one of the following: 

searching for said context information, 

displaying the context information simultaneously with the captured image 
pixels, and navigating a network to a source of the captured image pixels. 



3788428vl 



Page 7 of 23 



Application No. 10/766,319 306171.01/MFCP.153380 

Response Filed 03/03/2010 

Reply to Office Action of: 10/05/2009 

21. (Currently Amended) The computer-implemented method of claim 10, 
wherein the one or more graphical elements represent representing a first set of one or more 
textual characters, and the method further comprising: 

obtaining additional context information by extracting the first set of one 
or more textual characters, extracting a second set of textual characters displayed 
in proximity with the first set, and storing the first and second sets of textual 
characters as the additional context information, the additional context 
information being automatically stored in association with the image file. 

22. (Currently Amended) The computer-implemented method of claim 10, 
wherein the selected on-screen region is part of a displayed textual region, and the graphical 
elements comprise a first set of one or more textual characters displayed in the textual region, 
and the method further comprising: 

obtaining additional context information based on a second set of one or 
more textual characters displayed in the textual region, the additional context 
information being automatically stored in association with the image file. 

23. (Canceled) 

24. (Currently Amended) The computer-implemented method of claim 27, 
further comprising: 

digitizing movements of a stylus across the display in order to receive an 
annotation; and 
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obtaining additional context information based on the received annotation, 
the additional context information being automatically stored in association with 
the image file. 

25. (Currently Amended) The computer-implemented method of claim 10, 
wherein the selected on-screen region includes at least a portion of a displayed web page or 
document, and the method further comprises: 

using an application programming interface (API) to query an application 
for additional context information, the additional context information being 
automatically stored in association with the image file, the queried application 
causing the one or more graphical elements to be displayed. 

26. (Currently Amended) The computer-implemented method of claim 25, 
further comprising: obtaining a uniform resource identifier (URI) of the web page or document 
as the context information, the URI being obtained as a result of the query using the API. 

27. (Currently Amended) A computer-implemented method of context 
harvesting from selected content using a computer system having a processor, memory, and data 
storage subsystems, the method performed in a stylus based computer system comprising the 

receiving a path drawn on the display by a user via an input device a 
stylus , the drawn path defining the boundaries of a selected on-screen region of 
the display, the selected on-screen region comprising a plurality of pixels, a 
displayed [[the]] content of the selected on-screen region including both textual 



3788428vl 



Page 9 of 23 



Application No. 10/766,319 306171.01/MFCP.153380 

Response Filed 03/03/2010 

Reply to Office Action of: 10/05/2009 

data and underlying data comprising at least one of: an executable object, a file, or 
[[and]] a link to remote content; 

capturing the plurality of pixels of the on-screen region; 

storing the captured pixels as an image file such that the image file is 
representative of only those pixels of the display within the on screen region, 
wherein the content displayed within the on screen region includes textual data or 
underlying data comprising at least one of an executable object, a file, and a link 



automatically determining that the content displayed within the on screen 
region includes the textual data; 

in response to automatically determining that the displayed content of the 
on-screen region includes the textual data, automatically extracting a character or 
word from the textual data and extracting complete sentences based upon 
punctuation as context information via the computer system ; 

automatically determining that the displayed content of the on screen 
region includes the underlying data by using an application programming 
interface (API) to query an application window associated with the content of the 
selected on screen region; 

in response to determining that the displayed content of the on-screen 
region includes the underlying data, automatically extracting a property of the 
underlying data as additional context information via the computer system , the 
property comprising at least one of: a file name, a file identifier, a uniform 
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resource locator (URL), a uniform resource identifier (URI), a folder name, or 
[[and]] meta-data; and 

storing the extracted context information and additional context 
information in association with the image file via the data storage subsystem , such 
that the context information is accessible when viewing the image file. 



28. (Currently Amended) A c omputer-implemented method of context 
harvesting from selected content using a computer system having a processor, memory, and data 



display, the method comprising the steps of : 

setting a mode of operation to a content capture mode for interpreting 
stylus input from an input device for the purpose of selecting an on-screen region 



device a stylus , the drawn path defining the boundaries of a selected on-screen 
region of the display, the selected on-screen region comprising a plurality of 
pixels; 

capturing the plurality of pixels of the on-screen region; 

storing the captured pixels as an image file via the data storage subsystem 
such that the image file is representative of only those pixels of the display within 
the on screen region , wherein the content displayed within the on-screen region 
includes at least one of: textual data, an executable object, a file, or [[and]] a link 
to remote content; 



storage subsystems, the method performed ii 



stylus based computer system including 



of a display, and receiving a path drawn on the display by 



the input 
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receiving a user command to set the mode of operation to an annotation 
mode for interpreting stylus input for the purposes of annotating the content 
displayed within the onscreen region; 

in response to the user command, switching the mode of operation to an 
[[the]] annotation mode and receiving an annotation drawn on the display by the 
user via the input device stylus ; 

interpreting the input and annotating the content displayed within the on- 
screen region; 

performing text recognition on the annotation to produce recognized text 
of the annotation as context information; 

automatically determining via the computer system that the content 
displayed within the on-screen region includes at least one of textual data and 
underlying data comprising at least one of an executable object, a file, and a link 
to remote content; 

automatically extracting as additional context information at least one of : 
a bounded character or word from textual data determined to be 
included in the on-screen region , and determining if the bounded character or 
word lies within the selected on-screen region , and 

a property of underlying data determined to be included in the on- 
screen region, the property comprising at least one of: a file name, a file identifier, 
a uniform resource locator (URL), a uniform resource identifier (URI), a folder 
name, or [[and]] meta-data; and 
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storing the context information and the additional context information in 
association with the image file via the data storage subsystem , such that the 
context information is accessible when viewing the image file. 

29. (Canceled) 

30. (Currently Amended) The computer-implemented method of claim 10, 
wherein the annotation is stored as originally drawn as additional context information in 
association with the image file. 

3 1 . (Currently Amended) The computer-implemented method of claim 28, 
wherein the annotation is stored as originally drawn as additional context information in 
association with the image file. 

32. (New) The computer-implemented method of claim 10, further 
comprising a link to the one or more graphical elements of the selected on-screen region, the link 
comprising one of an implicit link hidden behind the one or more graphical elements and an 
explicit link shown in conjunction with the one or more graphical elements. 

33. (New) The computer-implemented method of claim 10, further 
comprising a separate linking table or database entry utilizing one or more pointers to the one or 
more graphical elements. 

34. (New) The computer-implemented method of claim 10, further 
comprising displaying the obtained context information to the user as one of displaying a 
property in a window and hovering over the one or more graphical elements. 
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35. (New) The computer-implemented method of claim 10, wherein the one 
or more graphical elements comprise underlying data, underlying objects, and underlying 
associations with remote content. 

36. (New) The computer-implemented method of claim 21, further 
comprising: determining that the second sets of textual characters reside within the path drawn 
by the user and any subsequent sets of textual characters reside outside the path drawn by the 
user. 

37. (New) The computer-implemented method of claim 10, wherein the 
received annotation is implemented using a plurality of tools via a toolbar, the toolbar appearing 
after a stylus has been lifted off the display. 
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